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ABSTRACT 



A technique for sharing a resource among a cluster of 
devices in a computer network. The technique involves 
generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum. The technique further involves, when the 
quorum is reached, selecting a most advanced generation 
indicator from among the voting devices, advancing the 
selected generation indicator and storing the advanced 
selected generation indicator in memory as a cluster gen- 
eration indicator. Upon the advanced selected generation 
indicator being stored in memory, the cluster is formed and 
includes the voting devices. The technique further involves 
sharing a resource among the voting devices after the cluster 
is formed. 
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COMPUTER NETWORK CLUSTER 
GENERATION INDICATOR 

FIELD OF THE INVENTION 

This invention relates generally to device clustering, and 
more particularly to sharing a resource among devices of a 
cluster. 

BACKGROUND OF THE INVENTION 

A cluster is a group of devices that are connected together 
in a network. Atypical cluster includes one or more devices 
that perform data processing operations (e.g., a mainframe), 
and one or more devices that do not perform data processing 
operations (e.g., a disk). Hie devices that perform data 
processing operations are considered processing devices or 
nodes, and the devices that do not perform data processing 
operations are considered non-processing devices. 

It is common for the processing devices of a cluster to 
share one or more resources. When the processing devices 
share a resource, each processing device cooperates with the 
other processing devices to coordinate use of the shared 
resource. For example, the processing devices of a particular 
cluster may share a storage disk. The processing devices 
may cooperate with each other by coordinating access to 
storage locations on the storage disk. 

Devices prefer to join clusters rather than form clusters. 
That is, when a processing device boots, the processing 
device attempts first to join a cluster if one exists, and if one 
does not exist, the processing device attempts alternatively 
to form a new cluster. Devices typically form a cluster by 
contributing votes to a cluster vote count. When the devices 
contribute enough votes so that the vote count reaches a 
threshold amount called a quorum, the devices form a 
cluster. Devices that contribute one or more votes are 
considered voting devices. Devices that contribute zero 
votes or devices that do not vote are considered non- voting 
devices. In general, a voting device of a cluster may be a 
processing device (e.g., a voting node) or a non-processing 
device (e.g., a quorum disk). 

Quorum will now be discussed in further detail. Cluster 
administrators typically set the quorum of a cluster equal to 
more than one half of the total number of votes available to 
the cluster. That is, the cluster administrator generates a total 
number of votes available to the cluster by counting the 
votes provided by all of the devices that are configured to 
form or join the cluster. Then, the cluster administrator sets 
the quorum equal to a value that is greater than one half the 
generated total number. 

The processing devices typically do not perform data 
processing operations until the cluster is formed, i.e., until 
the cluster vote count reaches quorum. Prior to formation of 
the cluster, the processing devices generally have limited 
operability such as the capability to boot, to contribute votes 
to the cluster vote count, and to transfer device information 
from one device to another. Once the cluster is formed, the 
processing devices perform data processing operations, and 
devices may join or leave the formed cluster. When a device 
joins the cluster, any votes of the joining device are added 
to the cluster vote count. Similarly, when a device leaves the 
formed cluster, any votes of the leaving device are removed 
from the cluster vote count. That is, the cluster recalculates 
the vote count when a device joins or leaves so that the vote 
count includes only the votes of the devices presently in the 
cluster. 

When one or more devices leave the cluster and the 
cluster vote count is recalculated, it is possible for quorum 
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to be lost. That is, the recalculated vote count may drop 
below quorum. When a cluster loses quorum, the devices 
remaining in the cluster enter a "hanging" state, and cease 
performing data processing operations. The devices do not 

5 resume data processing operations unless one or more 
devices join the cluster and contribute enough votes so that 
the vote count reaches quorum again. When quorum is 
restored to the cluster, the hanging processing devices 
complete any data processing operations that were left 

1Q uncompleted. 

Cluster partitioning will now be explained. A cluster may 
include two sets of devices that become disconnected from 
each other (e.g., due to a failure somewhere in the network). 
When disconnection occurs, each set becomes a separate 

15 cluster such that a first new cluster and a second new cluster 
exist simultaneously. That is, from the perspective of the first 
set of devices (the first new cluster), the devices of the 
second set have left the original cluster. Similarly, from the 
perspective of the second set (the second new cluster), the 

2Q devices of the first set have left. Such separation of the 
original cluster into multiple new clusters is called cluster 
partitioning. 

It should be understood that, in general, two partitioned 
clusters cannot perform data processing operations simulta- 

25 neously. In particular, since quorum equals more than one 
half of the total votes available to the original cluster, only 
one of the two new clusters can have quorum at any given 
time. If the first set of devices has enough votes to maintain 
quorum, the second set will not have enough votes for 

30 quorum. In this situation, the first set continues to perform 
data processing operations while the second set hangs. 
Alternatively, if the second set of devices has enough votes 
to maintain quorum, the second set continues to perform 
data processing operations while the first set hangs. After 

35 partitioning, it is possible that neither of the two new clusters 
has enough votes to maintain quorum such that both new 
clusters hang. 

Some clusters use generation numbers to track cluster 
operation. In such a cluster, the processing voting devices 

40 typically store a current cluster generation number in dedi- 
cated non-volatile memory locations (e.g, local non-volatile 
memory). When a processing voting device leaves the 
cluster, the remaining processing voting devices typically 
increment the cluster generation number, and store the 

45 incremented cluster generation number in place of the 
original cluster generation number The processing voting 
device that leaves typically does not increment its cluster 
generation number and continues to store the original cluster 
generation number. Accordingly, the processing voting 

50 devices that remain in the cluster should always have a 
cluster generation number that is greater than or equal to that 
of a processing voting device that has left the cluster. If a 
cluster's current cluster generation number is less than that 
of a processing voting device that has not joined the cluster, 

55 the cluster has become operational inadvertently, and the 
cluster should be stopped from operating as quickly as 
possible (e.g., crashed) to avoid destroying cluster informa- 
tion. 

When a processing voting device joins an already formed 
60 cluster, the processing voting device typically compares its 
stored generation number with the current cluster generation 
number (provided by a processing voting device of the 
cluster). If the stored generation number of the processing 
voting device is less than or equal to the current cluster 
65 generation number, the processing voting device typically 
joins the cluster and stores the current cluster generation 
number in place of its stored generation number. However, 
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if the stored genera tioo number of the processing voting 
device is greater than the current cluster generation number, 
the processing voting device sends a signal to the cluster 
indicating that its generation number is higher than the 
current generation number of the cluster. The processing 
devices of the cluster generally crash when the cluster 
receives the signal so that any damaging data processing 
operations performed by the processing devices of the 
cluster are discontinued. 

Examples of clusters include Open VMS clusters and 
UNIX TruClusters made by Digital Equipment Corporation 
of Maynard, Massachusetts, clusters made by Sequent Com- 
puter Systems, Inc. of Beaverton, Oregon, and clusters made 
by Tandem Computers of Austin, Texas. 

Some conventional clusters are susceptible to cluster 
partitioning into multiple new clusters. Subsequent opera- 
tion of such new clusters at different times can destroy 
cluster information on a shared resource in some situations. 

SUMMARY OF THE INVENTION 

Partitioning a conventional cluster into two new clusters 
that operate at different times may result in a loss of cluster 
information. In particular, the results of data processing 
operations, which are performed by one of the two new 
clusters and stored on a shared resource, may be overwritten 
by the results of data processing operations performed by the 
other new cluster. As will be discussed later, a preferred 
embodiment of the invention avoids such a loss of cluster 
information by associating generation indicators with cluster 
votes, and using the indicators to prevent such new clusters 
from operating at different times. 

First, a situation will be described in which a conventional 
cluster is partitioned into two new clusters that operate at 
different times such that cluster information on a shared 
resource is destroyed. When an original cluster is partitioned 
into a first new cluster and a second new cluster (e.g., due 
to a failure in the network), the first new cluster may retain 
quorum and continue performing data processing operations 
for a period of time, while the second new cluster loses 
quorum and b angs.^TOegdatafrprjo^ss^^ 
^rjdatgjda^a|orj|a^^ 

jdisk. Then, the first new cluster may lose quorum and the 
second new cluster may obtain quorum. For example, in the 
first new cloister, a processing device having access to the 
quorum disk may be shutdown such that the first new cluster 
loses quorum. Shortly thereafter, another processing device 
having access to the quorum disk may join the second new 
cluster so that the second new cluster obtains quorum. Once 
the second new cluster obtains quorum, the second new 
cluster begins performing data processing operations includ- 
ing completing any data processing operations that had been 
left uncompleted when partitioning of the original cluster 
occurred. Completion of the uncompleted operations may 
overwrite results generated by the first new cluster that 
reside on the quorum disk thus destroying cluster informa- 
tion. 

Cluster information can be lost even if cluster generation 
numbers are used. For example, suppose the processing 
device that attempts to join the second new cluster is a 
voting device having a stored generation number. If the 
device is in a "shutdown" state when partitioning of the 
original cluster occurs, the device should have a generation 
number that is less than that of the original cluster, and that 
of the two new clusters (since the cluster generation number 
is incremented when the shutdown device leaves the original 
cluster). When the device attempts to join the second new 
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cluster, the device compares its generation number with the 
cluster generation number of the second new cluster and 
determines that its generation number is less than that of the 
second new cluster. Accordingly, the device is permitted to 

5 join the second new cluster thereby allowing the second new 
cluster to regain quorum and overwrite cluster information 
generated by the first new cluster. 

In contrast, a preferred embodiment of the present inven- 
tion is directed to a technique for sharing a resource among 

10 a cluster of devices. The technique prevents cluster infor- 
mation from being destroyed when the cluster is partitioned 
into two new clusters where one of the new clusters main- 
tains quorum and continues to operate. The technique 
involves associating a generation indicator with each device 

15 that provides a vote. As such, if a new cluster resulting from 
a partitioned original cluster obtains votes that potentially 
allow it to regain quorum, the new cluster also obtains an 
associated generation indicator. The new cluster can deter- 
mine from that associated generation indicator that parti- 

20 tioning occurred and that the new cluster should not perform 
data processing operations that could damage cluster infor- 
mation. In particular, the new cluster crashes to prevent any 
subsequent damaging operations from occurring. 
In the preferred embodiment, the technique involves 

25 generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum. The quorum is preferably equal to more 
than one half of a total number of votes available to the 
cluster. When the quorum is reached, the technique involves 

30 writing a cluster generation indicator in a local memory of 
each voting device. At least one voting device is a non- 
processing voting device (e.g., a quorum disk). A cluster is 
formed that includes the voting devices when the cluster 
generation indicator is written in the local memory of each 

35 voting device. The technique further involves sharing a 
resource among the voting devices after the cluster is 
formed. 

Preferably, the generation indicator is advanced by the 
cluster whenever a device leaves and quorum is maintained, 

40 or whenever quorum is obtained. The generation indicator 
may be a number that is advanced by incremention. Any 
voting device wishing to join the cluster must compare the 
cluster generation indicator with its own generation indica- 
tor. If the cluster generation indicator is equal to or more 

45 advanced than its own generation indicator, the voting 
device is permitted to join the cluster. If not, the devices of 
the cluster crash. 

Since the cluster generation indicator is stored with each 
voting device, partitioning of the cluster into multiple new 

50 clusters does not allow subsequent operation of the new 
clusters at different times resulting in damaged or destroyed 
cluster information. Rather, if a first new cluster maintains 
quorum after partitioning, it advances the cluster generation 
indicator (since devices have left the cluster) and continues 

55 to perform data processing operations. The advanced gen- 
eration indicator is stored in each voting device of the first 
new cluster. If the first new cluster is shutdown and a voting 
device of the first new cluster (e.g., a quorum disk) attempts 
to join the second new cluster, the voting device must 

60 compare its generation indicator (i.e., the advanced genera- 
tion indicator) with the generation indicator of the second 
new cluster. The generation indicator of the second new 
cluster is less than the advanced generation indicator of the 
joining device. Accordingly, the second new cluster crashes, 

65 and cluster information is preserved. Hence, the second new 
cluster is prevented from performing data processing opera- 
tions that would overwrite cluster information. 
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In the present invention, the original cluster may include 
voting devices and non -voting devices. Writing the cluster 
generation indicator may involve storing the indicator in the 
local memory of each voting device and a local memory of 
at least one non -voting device. Writing the indicator to at 
least one non- voting device increases the availability of the 
indicator to devices wishing to obtain it. Alternatively, 
writing may involve storing the cluster generation indicator 
in the local memory of each voting device while excluding 
non-voting devices from storing the cluster generation indi- 
cator. This minimizes the number of store operations per- 
formed by the devices as a whole during cluster formation, 
and may allow the cluster to be formed more quickly. 

Preferably, when a voting device leaves the cluster and the 
quorum is maintained, the technique involves advancing the 
cluster generation indicator, storing the advanced cluster 
generation indicator in the local memory of each voting 
device remaining in the cluster, and preventing each voting 
device remaining in the cluster from further sharing the 
resource until the advanced cluster generation indicator is 
stored in the local memory of each voting device remaining 
in the cluster. Preventing the resource from being shared 
until the indicator is stored guarantees that each voting 
device has a correct generation indicator before data pro- 
cessing operations continue. The advanced indicator is an 
indication to devices that subsequently join the cluster that 
a device has left the cluster. 

According to another embodiment, the technique involves 
generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum. When the quorum is reached, the tech- 
nique involves selecting a most advanced generation indi- 
cator from among the voting devices, advancing the selected 
generation indicator and storing the advanced selected gen- 
eration indicator in memory as a cluster generation indicator. 
A cluster is formed that includes the voting devices when the 
advanced selected generation indicator is stored in memory. 
The technique further involves sharing a resource among the 
voting devices after the cluster is formed. 

Preferably, each voting device includes a local memory. 
As such, storing the advanced selected generation indicator 
in memory involves writing the indicator in the local 
memory of each voting device. In particular, the local 
memory of each voting device may include a non-volatile 
memory, and writing the indicator may involve storing the 
indicator in the non-volatile memory of the local memory of 
each voting device. Accordingly, the voting devices can 
retain the indicators even if they lose power. 

One or more of the voting devices may be a non- 
processing voting device, and writing the advanced selected 
generation indicator may involve storing the indicator in the 
local memory of each non-processing voting device. In 
particular, at least one of the non-processing voting devices 
may be a voting storage device (a quorum disk), and writing 
the indicator may include storing the indicator in the local 
memory of each voting storage device. 

Preferably, formation of the cluster is blocked until the 
advanced selected generation indicator is written into the 
local memory of each voting device. This guarantees that 
each voting device has a correct indicator before data 
processing operations begin. 

As in the preferred embodiment, the formed cluster may 
include voting devices and non- voting devices. Writing the 
advanced selected generation indicator may involve storing 
the advanced selected generation indicator in the local 
memory of each voting device and a local memory of at least 
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one non- voting device. Alternatively, writing the indicator 
may involve storing the advanced selected generation indi- 
cator in the local memory of each voting device while 
excluding non-voting devices from storing the indicator. 

5 Preferably, when a voting device leaves the cluster and the 
quorum is maintained, the technique involves advancing the 
cluster generation indicator, storing the advanced cluster 
generation indicator in memory, and preventing each voting 
device remaining in the cluster from further sharing the 

1° resource until the advanced cluster generation indicator is 
stored in memory. 

Furthermore, before the step of generating, the quorum is 
preferably set equal to more than one half of a total number 
of votes available to the cluster. 

35 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects, features and advantages 
of the invention will be apparent from the following more 
20 particular description of preferred embodiments of the 
invention, as illustrated in the accompanying drawings in 
which like reference characters refer to the same parts 
throughout the different views. The drawings are not nec- 
essarily to scale, emphasis instead being placed upon illus- 
25 trating the principles of the invention. 

FIGS. 1A-1E are block diagrams of devices that form 
clusters using conventional cluster formation techniques. 

FIG. 2 is a flow diagram of a method performed by voting 
devices when the voting devices initialize, according to the 
30 present invention. 

FIG. 3 is a flow diagram of a method performed by voting 
devices of a cluster after a cluster has been formed, accord- 
ing to the present invention. 
3S FIGS. 4A-4G are block diagrams of devices that form 
clusters, using the methods of FIGS. 2 and 3 according to the 
present invention. 

DETAILED DESCRIPTION OF THE 
INVENTION 

40 

FIG. 1A shows a conventional cluster 10 that includes a 
network connection 12, processing devices NODE A, 
NODE B and NODE C, and a quorum disk 19. NODE A 
connects with the network connection 12 through line 14. 

4S Similarly, NODE B and NODE C connect with the network 
connection 12 through line 16 and line 18, respectively. The 
processing devices communicate (e.g., contribute votes to 
form the cluster 10) through the network connection 12. 
NODE A further connects with the quorum disk 19 through 

50 line 20. Similarly, NODE B connects with the quorum disk 
19 through line 22. The quorum disk 19 is a resource that is 
shared among the processing devices. 

NODE A, NODE B and the quorum disk 19 are voting 
devices and have one vote each. NODE C is a non-voting 

SS processing device. The quorum disk 19 contributes its vote 
through either NODE A or NODE B. Thus, the total number 
of votes available is 3. Accordingly, quorum is set to be 
greater than one half the total number of votes. In this 
example, quorum is set equal to 2. Each processing device 

so further includes a generation number N that initially is equal 
to 1. 

As shown in FIG. 1A, the cluster 10 has quorum (Le., the 
vote count is 3, there being 1 vote from each of NODE A, 
NODE B and the quorum disk 19) and is fully operational, 
65 That is, each processing device (NODE A, NODE B and 
NODE C) is considered to be in a "running" state and is 
performing data processing operations. 
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Then for example, NODE B is shutdown, as shown in The sequence of events described above and shown in 

FIG. IB. The "X" through NODE B indicates that NODE B FIGS. 1A-1E results in the partitioning of a cluster 10 into 

is no longer performing data processing operations. When multiple new clusters 24 and 26, and subsequent operation 

NODE B is shutdown, the cluster 10 recalculates the vote of the new clusters 24 and 26 at different times. Such 

count to include votes only from devices remaining in the 5 operation can destroy cluster information residing on a 

cluster 10. Accordingly, the vote count equals 2(1 vote from snared resource (e.g., data on the quorum disk 19). In 

each of NODE A and the quorum disk 19), and quorum is particular, the first new cluster 24 can store data processing 

retained. Hence, the remaining processing devices, NODE A results on the quorum disk 19 at a first time (FIG. 1C). Then, 

and NODE C, continue to perform data processing opera- the second new cluster 26 can overwrite those results at a 

tions. later time (FIG. IE) thus destroying the data stored by the 

Additionally, when NODE B leaves the cluster 10, the first new cluster 24. 

cluster 10 increments the generation number N so that it For example, if the processing devices are banking 

equals 2. This is illustrated in FIG. IB with N being shown computers, NODE C may have been in the middle of 

as changed in value from 1 to 2. NODE B does not recording a particular transaction for a particular bank 

increment its generation number N which remains at 1 since 15 account on the quorum disk 19 when the failure in line 14 

NODE B is leaving the cluster. occurred. The particular transaction is left uncompleted 

Suppose that a failure occurs in line 14 so that NODE A since NODE C hangs (see FIG. 1C). The first new cluster 24 

loses its connection with the network connection 12, as including NODE A and the quorum disk 19 continues to 

shown in FIG. 1C. Such a failure causes the cluster 10 to record banking transactions such that banking service is not 

become partitioned into two new clusters. In particular, 20 interrupted. The particular transaction left uncompleted by 

NODE A and the quorum disk 19 are devices belonging to NODE C can be re-entered in NODE A so that the transac- 

a first new cluster 24, and NODE C is a device belonging to tion is recorded. Other transactions such as transactions for 

a second new cluster 26. That is, from the perspective of the same bank account can be performed at NODE A. When 

NODE A and the quorum disk 19, NODE C has left the NODE A is removed from service and the second new 

original cluster 10. Similarly, from the perspective of NODE ^ cluster 26 is put in service (see FIG. IE), NODE C com- 

C, NODE A and the quorum disk have left the original pletes uncompleted data processing operations including the 

cluster 10. uncompleted particular transaction. As a result, the bank 

When this or such partitioning occurs, the first new cluster account information on the quorum disk 19 is damaged (e.g. , 

24 recalculates a vote count. The vote count of the first new the correct bank account balance is overwritten or the same 

cluster 24 equals 2 (one vote from NODE A and one vote 30 transaction is recorded twice), 

from the quorum disk 19). Accordingly, quorum is main- In a preferred embodiment of the present invention, each 

tained and the cluster generation number N is incremented voting device includes a generation indicator that is used to 

from 2 to 3 to reflect the departure of NODE C from the track cluster operation. Unlike conventional clusters, each 

cluster, as illustrated in FIG. 1C. NODE A is allowed to non-processing voting device (e.g., a quorum disk) includes 

continue performing data processing operations since the 35 a generation indicator. According to the preferred 

cluster 24 has quorum. embodiment, if a new cluster results from a partitioned 

In a similar manner, the second new cluster 26 recalcu- original cluster and obtains votes that potentially allow it to 

lates a vote count. The vote count of the second new cluster regain quorum, the new cluster also obtains an associated 

26 equals zero. Accordingly, the second new cluster 26 does generation indicator. The new cluster determines from that 

not have quorum, and NODE C hangs. The "X" through 40 associated generation indicator that partitioning occurred 

NODE C indicates that NODE C no longer performs data and that the new cluster should not perform data processing 

processing operations. In particular, data processing opera- operations that could damage cluster information, 

tions that were in the process of being performed by NODE Accordingly, the new cluster crashes to prevent any subse- 

C when partitioning occurred remain uncompleted. quent damaging operations from occurring. 

Furthermore, NODE C does not increment its generation 45 FIG. 2 illustrates a method 30 used by voting devices as 

number N. they become available to form or join a cluster, according to 

Suppose then that NODE A is shutdown so that line 14 an embodiment of the present invention. The method 30 is 

can be repaired, and NODE B is rebooted so that services performed by processing voting devices such as a 

provided by the processing devices can continue (i.e., so that mainframe, and non-processing voting devices such as a 

data processing operations can be performed), as shown in 50 quorum disk. In step 32, the voting device initializes itself, 

FIG. ID. NODE B sees the second new cluster 26 and and proceeds to step 34. For example, if the voting device is 

attempts to join it. In particular, NODE B compares its a mainframe, it runs a booting procedure. In step 34, the 

generation number (N=l) with the generation number of the voting device determines whether a cluster all ready exists, 

second new cluster 26 (N=2). Since the generation number If a cluster all ready exists, the voting device proceeds to 

of NODE B is less than that of the second new cluster 26, 55 step 36 to attempt to join the existing cluster. If no cluster 

NODE B is allowed to join the cluster. Accordingly, NODE exists, the voting device proceeds to step 38 to attempt to 

B contributes its vote to the vote count of the second new form a cluster. 

cluster 26 so that the vote count equals 1. Similarly, the In step 36 (cluster all ready exists), the voting device 

quorum disk 19, which is connected with NODE B through presents its generation indicator (e.g., a generation number 

line 22, contributes its vote to the vote count so that the total 60 GEN#) and votes to the cluster, and proceeds to step 40. In 

vote count equals 2. When the vote count reaches 2, quorum step 40, the voting device determines whether the generation 

is achieved and the second new cluster 26 begins performing indicator of the voting device is less advanced than or equal 

data processing operations, as shown in FIG. IE. In to a cluster generation indicator of the existing cluster. If the 

particular, NODE C which had been in a hanging state generation indicator is less advanced than or equal to the 

re-enters a running state and completes the data processing 65 cluster's, step 40 proceeds to step 42. Otherwise, step 40 

operations that were left uncompleted when partitioning proceeds to step 44. In step 42, the voting device joins the 

occurred. cluster, and proceeds to step 46. In step 46, the cluster 
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including the voting device determines whether quorum is whether another device is leaving. If so, step 74 returns to 
regained. If quorum is regained, the cluster generation step 68 so that the votes of the other voting device are also 
indicator is advanced (e.g., the cluster generation number is deleted from the vote count. If multiple devices leave the 
incremented) in step 48. If quorum is not regained, the cluster, the cluster may loop through steps 68, 70 and 74 
cluster generation number is not advanced and step 48 is 5 multiple times. If no other device is leaving, step 74 pro- 
skipped. Next, in step 50, the voting device stores the cluster to step 80 . i n step 80, the cluster determines whether 
generation indicator as its generation indicator. In particular, a votiog device ^ j ommg the cluster. If not, step 80 returns 
the voting device stores the indicator in local non-volatile to step 74 If a device fa joining) stcp 80 proceeds to stcp g 2 . 

memory. „ . , In step 82, the cluster adds the votes of the joining device to 
In connection with step 38, which follows step 34 when 1Q the clustcr vote ^ and ds to stg 84 In st 84 ^ 
a cluster does not exist, the voting device generates a vote ^ duster dctcrmines whcther m ^ ined . If thc 
coun mcludmg its votes and votes of any otfier devices ^that yote fa stm below quorum, step 84 proceeds to step 74. 
are attempting to form a cluster, and proceeds to step 52. In T - . • j «t_ i , / a. i * 
step 52, the voting device determines whether quorum is If ^ uorum regamed the cluster advances the cluster 
reached. If not, the voting device returns to step 38 because generation indicator, and stores it m memory (e.g., mere- 
it must wait for one or more other voting devices to 15 ments the cluster generation number), and returns to step 62. 
contribute votes so that quorum is reached. If quorum is Optionally, non-voting devices may store generation indi- 
reached, step 52 proceeds to step 54. In step 54, the voting cators to increase the availability of the indicator to other 
device forms a cluster with the other devices. Furthermore, devices. If a non-voting device is configured to store a 
the voting device provides its generation indicator to the generation indicator in local memory, the non-voting device 
other devices so that the devices forming the cluster select 20 operates according to the method 30 (and method 60 when 
the most advanced generation indicator (e.g., the highest belonging to a cluster) in a manner similar to that of voting 
generation number). The most advanced generation indica- devices. In particular, non-voting devices that store genera- 
tor is advanced (e.g., the generation number is incremented), tj on indicators operate according to the method 30 in FIG. 
and the advanced generation number is considered the 2 , except that they contribute zero votes, 
cluster's generation indicator. Each voting device of the 05 T ( , , , , , . 
cluster stores the cluster generation indicator in local non- 11 should be understood that each voting device stores a 
volatile memory generation indicator, and operates according to the method 
In connection with step 44, which follows step 40 when 30 > md method 60 when ^ a cluster Accordingly, 
the voting device's generation indicator is more advanced ^■non^rocessmgy^ 
than the cluster's generation indicator, the voting device 3 ^P^ratesiaeeordii^toTUSelrSelh 

sends a signal to the cluster indicating that its r ^^^^^|^locaUv f storedigeneration»indicatoM(see steps 48 and 54). 
indicator is more advanced. The cluster responds by crash- Hence, if a cluster is partitioned mto two new clusters, the 
ing to prevent any damaging operations from continuing. movement of a non-processing voting device such as a 
Preferably, the voting devices subsequently reinitialize and quorum disk from one of the new clusters to another new 
proceed to step 34 to form a cluster again. 35 cluster carries ^ il a generation indicator. Thus, if the new 
FIG. 3 illustrates a method 60 describing the operation of ^f ter obtains votes from a voting device that earlier 
a cluster after it has been formed (i.e., after the cluster vote belonged to a different new cluster it also obtains that voting 
count has reached quorum), according to an embodiment of device L s ^ C1 ^ d generation indicator. The new cluster 
the present invention. In step 62, the cluster operates at *f s mdlc ^ T to dete ™ that partitioning occurred, 
quorum (i.e., the cluster performs data processing 40 ™ eQ ne ^ f usX&T <; rashes to Prevent unintended opera- 
operations), and proceeds to step 64. If a shutdown com- { ' on that would possibly destroy cluster information (e.g., 
mand is received by the cluster, in step 64, the cluster shuts data on tne c * uorum dlsk )* 

down. Otherwise, the cluster remains operational and pro- Tn^ feature of the invention will be more apparent when 
ceeds to step 66. In step 66, if a voting device is leaving, the described by way of example with reference to FIGS, 
cluster proceeds to step 68. If no voting device is leaving, the 45 4A-^4G. A cluster 90 includes a network connection 92, 
cluster proceeds to step 76, In step 68, the votes of the processing devices NODE A, NODE B and NODE C, and a 
leaving voting device are deleted from the cluster vote quorum disk 99. NODE A connects with the network con- 
count. In particular, the cluster recalculates the vote count to nection 92 through line 94. Similarly, NODE B and NODE 
include only votes from voting devices remaining in the C connect with the network connection 92 through line 96 
cluster, and proceeds to step 70. In step 70, the cluster 50 ^ line 98 > respectively. The processing devices commu- 
determines whether quorum is maintained. If so, the cluster nicate (e.g., contribute votes to form the cluster 90) through 
proceeds to step 72. If not, the cluster proceeds to step 74. me network connection 92. NODE A further connects with 
In step 72, when quorum is maintained after a voting me quorum disk 99 through line 100. Similarly, NODE B 
device leaves the cluster, the cluster advances the cluster connects with the quorum disk 99 through line 102. The 
generation indicator, and stores it in memory. For example, 55 q uorum , disk 99 15 a resource that is shared among the 
if the cluster generation indicator is a cluster generation processing devices. 

number, the cluster increments the number. Step 72 then Each processing device includes processing circuitry 120, 

proceeds to step 76 which determines whether a voting memory 122 and network circuitry 124. The memory 122 

device is joining the cluster. If no voting device is joining the includes local non- volatile memory (e.g., magnetic disk 

cluster, step 76 proceeds back to step 62. If a voting device eo memory). In one embodiment, the memory 122 of the 

joins the cluster, step 76 proceeds to step 78. In step 78, the processing devices includes one or more programs having 

cluster includes the votes of the joining voting device in the code for performing the methods 30 and 60 (see FIGS. 2 and 

cluster vote count, and proceeds to step 62, That is, the ^*^pIgp j anojto 
cluster recalculates the vote count to include the votes of a U_^fijeTirals^50'a^ ^ 
devices in the cluster including the joining device. ^^•rrBmttSelp^ 

In step 74, which follows step 70 when quorum is lost due Additionally, the quorum disk 99 includes processing 

to a voting device leaving the cluster, the cluster determines circuitry 126, and some local memory 128. The processing 
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circuitry 126 allows the quorum disk 99 to perform certain Similarly, the second new cluster 106 recalculates its 

functions such as a self-test, and diagnostics. The local cluster vote count when it detects NODE A and the quorum 

memory 128 is memory dedicated to quorum disk 99. disk 99 leaving the cluster (see FIG. 3, step 68). The 

«N®DEfA*N0BE«B»aT3a , ltfteT^^ recalculated cluster vote count of the second new cluster 106 

devices having one vote each. Each voting device stores a 5 is zero which is below quorum, so NODE C hangs (see FIG. 

cluster generation indicator G. By way of example, the 3, steps 70, 74, 80, 82 and 84). Note that the cluster 

cluster generation indicator G is a cluster generation num- generation indicator for NODE C is not advanced since the 

ber. NODE C is a non-voting processing device, NODE C second new cluster 106 loses quorum and hangs, 

optionally may include the cluster generation indicator G to Suppose that the cluster administrator shuts down the 

increase the availability of indicator G. The quorum disk 99 1Q cluster 104 so that the connection between NODE A and the 

contributes its vote through either NODE A or NODE B. The network connection 92 can be repaired. Also suppose that 

total number of votes available is 3 (1 vote from each of NODE B is rebooted. FIG. 4D illustrates this situation. After 

NODE A, NODE B and the quorum disk 99). Quorum is set NODE B initializes (see step 32 of FIG. 2), NODE B 

to be greater than one half the total number of votes. In this determines whether a cluster all ready exists (see step 34). 

example, quorum is set equal to 2. The cluster generation NODE B detects the existence of the second new cluster 

indicator G of each processing device is initially set equal to 15 106, and presents its generation indicator (G=l) to the 

1, and stored in each device's local memory 122A, 122B, second new cluster 106. Since the NODE B's generation 

122C and 128. indicator is less than that of the second new cluster 106 

As shown in FIG. 4A, the cluster 90 has quorum (i.e., the (0=2), NODE B joins the second new cluster 106 (see FIG. 

vote count is 3) and is fully operational (see step 62 in FIG. 2, steps 40 and 42). The second new cluster 106 now has a 

3). Accordingly, each processing device (NODE A, NODE 20 vote count of 1, and quorum is not achieved (see step 46 in 

B and NODE C) is in the "running" state and performs data FIG. 2, and steps 80, 82 and 84 in FIG. 3). Accordingly, 

processing operations. The voting devices (NODE A, NODE NODE B and NODE C hang, as shown in FIG. 4E. 

B and quorum disk 99) and NODE C (since NODE C Additionally, NODE B stores the cluster generation indica- 

includes a generation indicator) operate according to the tor (G=2) as its new cluster generation indicator (FIG. 2, step 

method 30 (see FIG. 2), and the cluster 90 operates accord- 25 50). 

ing to the method 60 (see FIG. 3). The quorum disk 99 is connected to NODE B through line 

At some time, NODE B for example is shutdown, as 102. Accordingly, the quorum disk 99 contributes its vote to 

shown in FIG. 4B. When NODE B is shutdown, NODE B the second new cluster 106 as well. In particular, the quorum 

leaves the cluster 90 (see step 66 in FIG. 3), and the cluster disk initializes (FIG. 2, step 32), sees that a cluster 106 all 

90 recalculates the cluster vote count to include votes only 30 readv exists 2 > ste P 34 ) md presents its vote and 

from devices remaining in the cluster 90 (see step 68 in FIG. generation indicator to the second new cluster 106 (FIG. 2, 

3). The "X" through NODE B in FIG. 4B indicates that step 36). The second new cluster 106 determines that the 

NODE B no longer performs data processing operations, generation indicator G of the quorum disk 99 (G»3) is more 

The recalculated vote count equals 2 (1 from each of Node advanced than that of the second new cluster (G-2). 

A and Q Disk 99), and quorum is retained. As a result, the 35 Accordingly, quorum disk 99 crashes the cluster devices, 

cluster generation indicator is advanced such that G in namely NODE B and NODE C, as shown in FIG. 4F. Any 

NODE A, NODE C and the quorum disk 99 equals 2 (see operations that were uncompleted by NODE C when the 

step 72 in FIG. 3). In particular, the generation indicator is original cluster 90 was partitioned are lost, rather than 

advanced and stored in the local memory 122A, 122C and completed. Hence, there is no overwriting of cluster infor- 

128 of each remaining voting device and optionally the 40 mauoo residing on the shared resource (quorum disk 99) by 

remaining non-voting devices, and data processing opera- the second new cluster 106 devices (NODE B and NODE 

tions are blocked until such storing is complete. Once P)- 

storage of the indicator is complete, the remaining process- It should be understood that, unlike the situation in FIGS, 

ing devices, NODE A and NODE C, continue to perform 1A-1E where no generation indicator is associated with the 

data processing operations. NODE B does not increment its 45 vote of the quorum disk 19, a generation indicator is 

generation number G which remains at 1 since NODE B has associated with the vote of the quorum disk 99 (see FIGS, 

left the cluster 90. 4A-4G). Accordingly, the second new cluster 106 is able to 

Next in the example, the connection between NODE A determine that cluster partitioning has occurred and that a 

and the network connection 92 fails. For example, the cluster other than the second new cluster 106 had operated 

network adaptor 124A malfunctions, or line 94 becomes 50 usin g &e quorum disk 99 prior to the quorum disk 99 

disconnected, as shown in FIG. 4C. At this time, the cluster becoming available to the second new cluster 106. Once this 

90 becomes partitioned into two new clusters, namely a first situation is detected, the second new cluster 99 crashes (see 

new cluster 104 and a second new cluster 106. That is, from FIG - 4F ) to avoid running inadvertently (as shown in FIG. 

the perspective of NODE A and the quorum disk 99, NODE IE). 

C leaves the original cluster 90. The first new cluster 55 Although the example describes NODE B attempting to 
responds to the partitioning of the cluster 90 by deleting any join the second new cluster 106 before the quorum disk 99, 
votes contributed by NODE C (see step 68 in FIG. 3). Since both devices could attempt to join together, or the quorum 
NODE C contributes zero votes, no votes are deleted from disk 99 could attempt to join first. The same end result would 
the cluster vote count. That is, the cluster vote count includes be reached in either situation. In particular, it would be 
all the votes of the devices currently in the cluster, namely so determined that the generation indicator of the quorum disk 
one vote from NODE A and one vote from the quorum disk 99 was more advanced than that of the second new cluster 
99. Accordingly, the cluster vote count for the first new 106. Accordingly, the second new cluster 106 would crash to 
cluster 104 is 2 and quorum is retained. As a result, the remove any possibility of performing data processing opera- 
cluster generation indicator G of each device (NODE A and tions that would destroy cluster information on the shared 
the quorum disk 99) is advanced (i.e., the cluster generation 65 resource. 

number G is incremented to 3), and stored in each device's NODE B and NODE C may then re-initialize (reboot) and 

local memory (i.e., local memory 122A and 128). form a different new cluster 108 and resume data processing 
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operations without risk of damaging cluster information, as structure shown in FIGS. 4A-4G. Such modifications to the 

shown in FIG. G. That is, NODE B initializes and deter- structure shown in FIGS. 4A-4G are intended to be within 

mines that a cluster 108 now consists only of the quorum the scope of the invention. 

disk 99 (FIG. 2, steps 32 and 34). NODE B presents its Furthermore, the resource that is shared among cluster 

generation number which is currently G=2. The cluster 108 5 devices in the hardware configuration shown in FIGS, 

compares the cluster generation number (G=3 from the 4A-4G is anon-processing voting device (quorum disk 99). 

quorum disk 99) to NODE B*s generation number (G=2) The shared resource does not need to be a voting device. A 

and allows NODE B to join the cluster 108 (FIG. 2, step 42) non-voting resource may be shared as well. For example, the 

since NODE B's generation number is less than the cluster resource may be a non-voting device such as a storage disk 

generation number. The votes are counted to see if quorum 10 without votes. The invention prevents multiple new clusters 

is regained (FIG. 2, step 46). In this case, the vote count resulting from an original partitioned cluster from operating 

equals 2, there being 1 vote from the quorum disk 99 and 1 at different times on such a resource to prevent cluster 

vote from NODE B. Thus, quorum is regained and the information from bemg destroyed 

generation number is incremented from 3 to 4 (FIG. 2, step What is claimed is: - JJ? l • 

ao\ TU i nn i KTA nr n r i_ \ *u 1- IQ a computer network, a method tor sharing a resource 

48). The quorum disk 99 and NODE B (i.e.. each of the -i« , 5 c j \^ j 

, . 4 . t . , " \ * " . . 15 among a cluster of devices, the method comprising the steps 

devices in the currently existing cluster) store G=4 m 0 £ 

respective local memories 122B I 128 (FIG 2 step 50). ' ^ M ^ mc , udes vQtes from wti 

Similarly, NODE C joins cluster 108 by asserting a genera- devices attempting to form a cluster until the vote counl 

tion number (G=2) which is less than the cluster generation reaches a Quorum* 

number which equals 4 at this time (FIG. 2, steps 34, 36, 40 on <• J . j- 4 t • 4l _ 

and 42). NODE C stores G=4 in local memory 122C (FIG. gyrating a generation mdicator and storing the genera- 

2, step 50) and cluster 108 operates with NODE B and U ° D m * cator m each V^mg and non-processmg 

N0D E C VOtmg devlce; 

Alternatively, the quorum disk 99 alone may not belong when th *W°™*. is > rca f ched - * most advanced 

to any cluster when NODE B and NODE C crash. In this 25 g« n6rat >°° bom among the voung devices, 

situation, NODE B, NODE C and the quorum disk 99 advancmg the selected generation micator and storing 

contribute votes to a vote count (1 vote from NODE B, zero ad , va f < xd selec , t . ed g^on mdicator m memory 

votes from NODE C, and 1 vote from the quorum disk 99) f a ' lus '" S^hon indicator, forming a cluster that 

that equals 2 (FIG. 2, step 38). Then, the devices determine mclud ^ the ^ advanced Sel u 6C f ed 

that quorum is reached (FIG. 2, step 52), and form the cluster 30 gfneraUon indicator being ; stored in memory, such that 

ino/rrT^ 1 * ca\ ■rn, ^ • c a *u u- u * the cluster generation indicators are associated with 

108 (FIG. 2, step 54). The devices find the highest genera- . A . j 1 . . 1 1 A * , . 

fl - „ llm L Q , a-^u qq c* ~ i Z i\ Jj« ~ cluster votes and the cluster is able to determine from 

tion number (the quorum disk 9y has u equal to 3), advance . A , A . . Jt , A . A Al . A , 

w u * ♦ • — u fr* a\ a * *u u- u * the selected generation indicator that the cluster has 

the highest generation number (G=4), and store the highest , . . * 

generation number in their respective memories (122B, , bCCD P artltloned i and 

122C and 128) 35 sharing a resource among the votmg devices after such 

Thus, the present invention provides a method and appa- - ^ Ster ^f/ 0 ! 111 ^*. . A . ... 

* f * • j -1 . j i 4. I,. . . 2. Ine method ol claim 1, wherein each votine device 

ratus for stoppmg a partitioned cluster such that processing . . , , , • * • . « - * A . 

. , . ,. , * . ■ . . . mcludes a local memory, and where in the step of selecting, 

operations are not performed which would destroy cluster , . r B 

. \. if j advancing and storing includes the step of: 

information on a shared resource. & & r 

EQUIVALENTS 40 wnting the advanced selected generation indicator in the 

While this invention has been particularly shown and „ local memory of each voting device 

described with references to preferred embodiments thereof, 3 ; The method of claim 2, wherein the local memory of 

it will be understood by those skilled in the art that various ea L ch voi ^ devic ? mcludes a , ™n-volatile memory, and 

changes in form and details may be made therein without wherem me ste P of writl °g mcludes me ste P of: 

departing from the spirit and scope of the invention as 45 storing the advanced selected generation indicator in the 

defined by me appended claims. Those skilled in the art will non-volatile memory of the local memory of each 

recognize or be able to ascertain using no more than routine voting device. 

experimentation, many equivalents to the specific embodi- 4 - ^ method of claim 2, wherein at least one of the 

ments of the invention described specifically herein. Such votin S devices is a non-processing voting device, and 

equivalents are intended to be encompassed in the scope of 50 wherein the step of writing includes the step of: 

the claims. storing the advanced selected generation indicator in the 

For example, the generation indicator was described in l° c al memory of each non-processing voting device, 
terms of a number that is advanced through incrementation. 5 - The method of claim 4, wherein at least one non- 
Advancement through decrementation is also suitable. processing voting device is a voting storage device, and 
Alternatively, the generation indicator can be an ordered 55 wherein the step of storing the advanced selected generation 
sequence of symbols or codes that is advanced by changing indicator in the local memory of each non-processing voting 
the current symbol or code to the next symbol or code in the device includes the step of: 

sequence. storing the advanced selected generation indicator in the 

Additionally, hardware configurations other than that local memory of each voting storage device, 

shown in FIGS. 4A-4G are suitable for the invention. For 60 6. The method of claim 2, wherein the step of selecting, 

example, the processing devices in FIGS. 4A-4G are shown advancing and storing further includes the step of: 

as general purpose computers. The processing devices may blocking formation of the cluster until the advanced 

be other types of computing devices such as super- selected generation indicator is written into the local 

computing devices, distributed systems, and specialized memory of each voting device, 

transaction devices (e.g., banking or teller machines). 65 7. The method of claim 2, wherein the cluster includes 

Similarly, the network connection 92 may have topologies voting devices and non-voting devices, and wherein the step 

(e.g., token ring or mesh) other than the simple backbone of writing includes the step of: 
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storing the advanced selected generation indicator in the 
local memory of each voting device while excluding 
noD-voting devices from storing the advanced selected 
generation indicator. 

8. The method of claim 2, wherein the cluster includes 
voting devices and non-voting devices, and wherein the step 
of writing includes the step of: 

storing the advanced selected generation indicator in the 
local memory of each voting device and a local 
memory of at least one non-voting device. 

9. The method of claim 1, further comprising the step of: 
when a voting device leaves the cluster and the quorum is 

maintained, advancing the cluster generation indicator, 
storing the advanced cluster generation indicator in 
memory, and preventing each voting device remaining 
in the cluster from further sharing the resource until the 
advanced cluster generation indicator is stored in 
memory. 

10. In a computer network, a method for sharing a 
resource among a cluster of devices, the method comprising 
the steps of: 

generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum; 

when the quorum is reached, writing a cluster generation 
indicator in a local memory of each voting device, at 
least one voting device being a non-processing voting 
device, a cluster being formed that includes the voting 
devices when the cluster generation indicator is written 
in the local memory of each voting device, such that the 
cluster generation indicators are associated with cluster 
votes and the cluster is able to determine from a 
selected generation indicator that the cluster has been 
partitioned; and 

sharing a resource among the voting devices after the 
cluster is formed. 

11. The method of claim 10, wherein the cluster includes 
voting devices and non-voting devices, and wherein the step 
of writing includes the step of: 

storing the cluster generation indicator in the local 
memory of each voting device while excluding non- 
voting devices from storing the cluster generation indi- 
cator. 

12. The method of claim 10, wherein the cluster includes 
voting devices and non-voting devices, and wherein the step 
of writing includes the step of: 

storing the cluster generation indicator in the local 
memory of each voting device and a local memory of 
at least one non -voting device. 

13. The method of claim 10, further comprising the step 

of: 

when a voting device leaves the cluster and the quorum is 
maintained, advancing the cluster generation indicator, 
storing the advanced cluster generation indicator in the 
local memory of each voting device remaining in the 
cluster, and preventing each voting device remaining in 
the cluster from further sharing the resource until the 
advanced cluster generation indicator is stored in the 
local memory of each voting device remaining in the 
cluster. 

14. The method of claim 10, further comprising: 
before the step of generating, setting the quorum equal to 

more than one half of a total number of votes available 
to the cluster. 

15. In a computer network, a system for sharing a resource 
among a cluster of devices, comprising: 
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a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes respective votes 

5 from voting devices attempting to form a cluster 

until the vote count reaches a quorum; 
generating a generation indicator and storing the gen- 
eration indicator in each processing and non- 
processing voting device; 

10 when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices 
that attempted to form the cluster, advancing the 
selected generation indicator and storing the 
advanced selected generation indicator in the local 

15 memory of at least one voting device as a cluster 

generation indicator, a cluster being formed that 
includes the voting devices that attempted to form 
the cluster when the advanced selected generation 
indicator is stored in the local memory of the at least 

20 one voting device, such that the cluster generation 

indicators are associated with cluster votes and the 
cluster is able to determine from the selected gen- 
eration indicator that the cluster has been partitioned; 
and 

25 sharing the shareable resource among the at least one 
voting device after the cluster is formed. 

16. The system of claim 15, wherein the code includes 
instructions for storing the advanced selected generation 
indicator in the local memory of each voting device that 

30 attempted to form the cluster, the cluster being formed when 
the advanced selected generation indicator is stored in the 
local memory of each voting device that attempted to form 
the cluster. 

17. The system of claim 16, wherein the local memory of 
35 each voting device includes a non-volatile memory, and 

wherein the instructions for storing include instructions for 
writing the advanced selected generation indicator in the 
non-volatile memory of the local memory of each voting 
device. 

40 18. The system of claim 16, wherein at least one of the 
voting devices is a non-processing voting device, and 
wherein the instructions for storing includes instructions for 
writing the advanced selected generation indicator in the 
local memory of each non-processing voting device. 

45 19. The system of claim 18, wherein at least one non- 
processing voting device is a voting storage device, and 
wherein the instructions for writing the advanced selected 
generation indicator in the local memory of each non- 
processing voting device includes instructions for storing the 

50 advanced selected generation indicator in the local memory 
of each voting storage device. 

20. The system of claim 16, wherein the code includes 
instructions for blocking formation of the cluster until the 
advanced selected generation indicator is written into the 

55 local memory of each voting device. 

21. The system of claim 16, further comprising: 

a plurality of non-voting devices, wherein the cluster 
includes voting devices and non-voting devices, and 
wherein the code includes instructions for storing the 
eo advanced selected generation indicator in the local 
memory of each voting device while excluding non- 
voting devices from storing the advanced selected 
generation indicator. 

22. The system of claim 16, further comprising: 

65 a plurality of non-voting devices, wherein the cluster 
includes voting devices and non- voting devices, and 
wherein the code includes instructions for storing the 
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advanced selected generation indicator in the local 
memory of each voting device and a local memory of 
at least one non-voting device. 

23. The system of claim 15, wherein the code includes 
instructions for, when a voting device leaves the cluster and 5 
the quorum is maintained, advancing the cluster generation 
indicator, storing the advanced cluster generation indicator 

in memory, and preventing each voting device remaining in 
the cluster from further sharing the resource until the 
advanced cluster generation indicator is stored in memory. 

24. In a computer network, a system for sharing a resource 
among a cluster of devices, comprising: 

a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 15 
generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote 
count reaches a quorum; 
when the quorum is reached, writing a cluster genera- 
tion indicator in the local memory of each voting 20 
device that attempted to form the cluster, at least one 
voting device being a non-processing voting device, 
a cluster being formed that includes the voting 
devices that attempted to form the cluster when the 
cluster generation indicator is written in the local ^ 
memory of each voting device that attempted to form 
the cluster; and 
sharing the shareable resource among the voting 
devices that attempted to form the cluster after the 
cluster, such that the cluster generation indicators are 30 
associated with cluster votes and the cluster is able to 
determine from the selected generation indicator that 
the cluster has been partitioned is formed. 

25. The system of claim 24, further comprising: 

a plurality of non-voting devices wherein the cluster 35 
includes voting devices and non-voting devices, and 
wherein the code includes instructions for storing the 
cluster generation indicator in the local memory of each 
voting device while excluding non-voting devices from 
storing the cluster generation indicator. 40 

26. The system of claim 24, further comprising: 

a plurality of non-voting devices, wherein the cluster 
includes voting devices and non-voting devices, and 
wherein the code includes instructions for storing the 
cluster generation indicator in the local memory of each 45 
voting device and a local memory of at least one 
non-voting device. 

27. The system of claim 24, wherein the code includes 
instructions for, when a voting device leaves the cluster and 
the quorum is maintained, advancing the cluster generation 50 
indicator, storing the advanced cluster generation indicator 

in the local memory of each voting device remaining in the 
cluster, and preventing each voting device remaining in the 
cluster from further sharing the resource until the advanced 
cluster generation indicator is stored in the local memory of 55 
each voting device remaining in the cluster. 

28. The system of claim 24, wherein the code includes 
instructions for setting the quorum equal to more than one 
half of a total number of votes available to the cluster before 
generating the vote count. eo 

29. In a computer network, a method for sharing a 
resource among a cluster of devices, the cluster including at 
least one non-processing voting storage device, the method 
comprising the steps of: 

generating a vote count that includes votes from voting 65 
devices attempting to form a cluster until the vote count 
reaches a quorum; 
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generating a generation indicator and storing the genera- 
tion indicator in each voting device; 

when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices, 
advancing the selected generation indicator, storing the 
advanced selected generation indicator in memory as a 
cluster generation indicator, a cluster being formed that 
includes the voting devices when the advanced selected 
generation indicator is stored in memory, and storing 
the advanced selected generation indicator in a local 
memory of each non-processing voting storage device, 
such that the cluster generation indicators are associ- 
ated with cluster votes and the cluster is able to 
determine from the selected generation indicator that 
the cluster has been partitioned; and 

sharing a resource among the voting devices after the 
cluster is formed. 

30. In a computer network, a method for sharing a 
resource among a cluster of devices, the cluster including 
voting devices and non-voting devices, the method com- 
prising the steps of: 

generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum; 

generating a generation indicator and storing the genera- 
tion indicator in each voting device; 

when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices, 
advancing the selected generation indicator, storing the 
advanced selected generation indicator in memory as a 
cluster generation indicator, a cluster being formed that 
includes the voting devices when the advanced selected 
generation indicator is stored in memory, and storing 
the advanced selected generation indicator in a local 
memory of each voting device while either a) excluding 
non-voting devices from storing the advanced selected 
generation indicator or b) storing the advanced selected 
generation indicator in the local memory of at least one 
non-voting device, such that the cluster generation 
indicators are associated with cluster votes and the 
cluster is able to determine from the selected generation 
indicator that the cluster has been partitioned; and 

sharing a resource among the voting devices after the 
cluster is formed. 

31. In a computer network, a method for sharing a 
resource among a cluster of devices, the method comprising 
the steps of: 

generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum; 

generating a generation indicator and storing the genera- 
tion indicator in each voting device; 

when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices, 
advancing the selected generation indicator and storing 
the advanced selected generation indicator in memory 
as a cluster generation indicator, a cluster being formed 
that includes the voting devices when the advanced 
selected generation indicator is stored in memory, such 
that the cluster generation indicators are associated 
with cluster votes and the cluster is able to determine 
from the selected generation indicator that the cluster 
has been partitioned; 

when a voting device leaves the cluster and the quorum is 
maintained, advancing the cluster generation indicator, 
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storing the advanced cluster generation indicator in 
memory, and preventing each voting device remaining 
in the cluster from further sharing the resource until the 
advanced cluster generation indicator is stored in 
memory; and 5 
sharing a resource among the voting devices after the 
cluster is formed. 

32. In a computer network, a method for sharing a 
resource among a cluster of devices, the cluster including 
voting devices and non-voting devices, the method com- 10 
prising the steps of: 

generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote count 
reaches a quorum; 

when the quorum is reached, writing a cluster generation 
indicator in a local memory of each voting device, at 
least one voting device being a non-processing voting 
device, while either a) excluding non-voting devices 
from storing the cluster generation indicator or b) 
storing the advanced selected generation indicator in 
the local memory of at least one non-voting device, a 
cluster being formed that includes the voting devices 
when the cluster generation indicator is written in the 
local memory of each voting device, such that the 
cluster generation indicators are associated with cluster 
votes and the cluster is able to determine from a 
selected generation indicator that the cluster has been 
partitioned; and 

sharing a resource among the voting devices after the 30 
cluster is formed. 

33. In a computer network, a method for sharing a 
resource among a cluster of devices, the method comprising 
the steps of: 

generating a vote count that includes votes from voting 35 
devices attempting to form a cluster until the vote count 
reaches a quorum; 

when the quorum is reached, writing a cluster generation 
indicator in a local memory of each voting device, at 
least one voting device being a non-processing voting 40 
device, a cluster being formed that includes the voting 
devices when the cluster generation indicator is written 
in the local memory of each voting device; 

when a voting device leaves the cluster and the quorum is 
maintained, advancing the cluster generation indicator, 
storing the advanced cluster generation indicator in the 
local memory of each voting device remaining in the 
cluster, and preventing each voting device remaining in 
the cluster from further sharing the resource until the 
advanced cluster generation indicator is stored in the 
local memory of each voting device remaining in the 
cluster such that the cluster generation indicators are 
associated with cluster votes and the cluster is able to 
determine from a selected generation indicator that the 
cluster has been partitioned; and 

sharing a resource among the voting devices after the 
cluster is formed. 

34. In a computer network, a system for sharing a resource 
among a cluster of devices, the cluster including at least one 
non-processing voting storage device, comprising: 

a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes respective votes 65 

from voting devices attempting to form a cluster 

until the vote count reaches a quorum; 
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generating a generation indicator and storing the gen- 
eration indicator in each voting device; 

when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices 
that attempted to form the cluster, advancing the 
selected generation indicator and storing the 
advanced selected generation indicator in the local 
memory of each non-processing voting storage 
device that attempted to form the cluster, as a cluster 
generation indicator, a cluster being formed that 
includes the voting devices that attempted to form 
the cluster when the advanced selected generation 
indicator is stored in the local memory of each voting 
device that attempted to form the cluster, such that 
the cluster generation indicators are associated with 
cluster votes and the cluster is able to determine from 
the selected generation indicator that the cluster has 
been partitioned; and 

sharing the shareable resource among the at least one 
voting device after the cluster is formed. 

35. In a computer network, a system for sharing a resource 
among a cluster of devices, the cluster including voting and 
non-voting devices and at least one non-processing voting 
storage device, comprising: 

a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes respective votes 
from voting devices attempting to form a cluster 
until the vote count reaches a quorum; 
generating a generation indicator and storing the gen- 
eration indicator in each voting device; 
when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices 
that attempted to form the cluster, advancing the 
selected generation indicator and storing the 
advanced selected generation indicator in the local 
memory of each voting device that attempted to form 
the cluster, as a cluster generation indicator, while 
either a) excluding non-voting devices from storing 
the advanced selected generation indicator or b) 
storing the advanced selected generation indicator in 
the local memory of each voting device and a local 
memory of at least one non-voting device, a cluster 
being formed that includes the voting devices that 
attempted to form the cluster when the advanced 
selected generation indicator is stored in the local 
memory of each voting device that attempted to form 
the cluster, such that the cluster generation indicators 
are associated with cluster votes and the cluster is 
able to determine from the selected generation indi- 
cator that the cluster has been partitioned; and 
sharing the shareable resource among the at least one 
voting device after the cluster is formed. 

36. In a computer network, a system for sharing a resource 
among a cluster of devices, comprising: 

a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes respective votes 
from voting devices attempting to form a cluster 
until the vote count reaches a quorum; 
generating a generation indicator and storing the gen- 
eration indicator in each voting device; 
when the quorum is reached, selecting a most advanced 
generation indicator from among the voting devices 
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that attempted to form the cluster, advancing the 
selected generation indicator and storing the 
advanced selected generation indicator in the local 
memory of at least one voting device as a cluster 
generation indicator, a cluster being formed that 
includes the voting devices that attempted to form 
the cluster when the advanced selected generation 
indicator is stored in the local memory of the at least 
one voting device, such that the cluster generation 
indicators are associated with cluster votes and the 
cluster is able to determine from the selected gen- 
eration indicator that the cluster has been partitioned; 
when a voting device leaves the cluster and the quorum 
is . maintained, advancing the cluster generation 
indicator, storing the advanced cluster generation 
indicator in memory, and preventing each voting 
device remaining in the cluster from further sharing 
the resource until the advanced cluster generation 
indicator is stored in memory; and 
sharing the shareable resource among the at least one 
voting device after the cluster is formed. 
37. In a computer network, a system for sharing a resource 
among a cluster of devices, the cluster including voting 
devices and non-voting devices, comprising: 
a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote 
count reaches a quorum; 
when the quorum is reached, writing a cluster genera- 
tion indicator in the local memory of each voting 
device that attempted to form the cluster, at least one 
voting device being a non-processing voting device, 
while either a) excluding non-voting devices from 
storing the cluster generation indicator or b) storing 
the cluster generation indicator in the local memory 
of each voting device and a local memory of at least 
one non-voting device, a cluster being formed that 
includes the voting devices that attempted to form 
the cluster when the cluster generation indicator is 
written in the local memory of each voting device 
that attempted to form the cluster, such that the 
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cluster generation indicators are associated with 
cluster votes and the cluster is able to determine from 
a selected generation indicator that the cluster has 
been partitioned; and 
sharing the shareable resource among the voting 
devices that attempted to form the cluster after the 
cluster is formed. 
38. In a computer network, a system for sharing a resource 
among a cluster of devices, comprising: 
a shareable resource; and 

a plurality of voting devices, each voting device having a 
local memory, the voting devices including code for: 
generating a vote count that includes votes from voting 
devices attempting to form a cluster until the vote 
count reaches a quorum; 
when the quorum is reached, writing a cluster genera- 
tion indicator in the local memory of each voting 
device that attempted to form the cluster, at least one 
voting device being a non-processing voting device, 
a cluster being formed that includes the voting 
devices that attempted to form the cluster when the 
cluster generation indicator is written in the local 
memory of each voting device that attempted to form 
the cluster; 

when a voting device leaves the cluster and the quorum 
is maintained, advancing the cluster generation 
indicator, storing the advanced cluster generation 
indicator in the local memory of each voting device 
remaining in the cluster, and preventing each voting 
device remaining in the cluster from further sharing 
the resource until the advanced cluster generation 
indicator is stored in the local memory of each voting 
device remaining in the cluster, such that the cluster 
generation indicators are associated with cluster 
votes and the cluster is able to determine from a 
selected generation indicator that the cluster has been 
partitioned; and 

sharing the shareable resource among the voting 
devices that attempted to form the cluster after the 
cluster is formed. 
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